Building an image classification model using very little data

Based on the tutorial by Francois Chollet @fchollet https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html and the workbook by Guillaume Dominici https://github.com/gggdominici/keras-workshop

This tutorial presents several ways to build an image classifier using keras from just a few hundred or thousand pictures from each class you want to be able to recognize.

We will go over the following options:

  • training a small network from scratch (as a baseline)
  • using the bottleneck features of a pre-trained network
  • fine-tuning the top layers of a pre-trained network

This will lead us to cover the following Keras features:

  • fit_generator for training Keras a model using Python data generators
  • ImageDataGenerator for real-time data augmentation
  • layer freezing and model fine-tuning
  • ...and more.

Data

Data can be downloaded at: https://www.kaggle.com/c/dogs-vs-cats/data
All you need is the train set
The recommended folder structure is:

Folder structure

data/
    train/
        dogs/ ### 1024 pictures
            dog001.jpg
            dog002.jpg
            ...
        cats/ ### 1024 pictures
            cat001.jpg
            cat002.jpg
            ...
    validation/
        dogs/ ### 416 pictures
            dog001.jpg
            dog002.jpg
            ...
        cats/ ### 416 pictures
            cat001.jpg
            cat002.jpg
            ...

Note : for this example we only consider 2x1000 training images and 2x400 testing images among the 2x12500 available.

The github repo includes about 1500 images for this model. The original Kaggle dataset is much larger. The purpose of this demo is to show how you can build models with smaller size datasets. You should be able to improve this model by using more data.

Data loading


In [1]:
##This notebook is built around using tensorflow as the backend for keras
#!pip install pillow
!KERAS_BACKEND=tensorflow python -c "from keras import backend"


Using TensorFlow backend.

In [8]:
import os
import numpy as np
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.preprocessing.image import ImageDataGenerator
from keras.layers import Conv2D, Convolution2D, MaxPooling2D, ZeroPadding2D
from keras import optimizers

In [9]:
# dimensions of our images.
img_width, img_height = 150, 150

train_data_dir = 'data/train'
validation_data_dir = 'data/validation'

Imports


In [10]:
# used to rescale the pixel values from [0, 255] to [0, 1] interval
datagen = ImageDataGenerator(rescale=1./255)

# automagically retrieve images and their classes for train and validation sets
train_generator = datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=16,
        class_mode='binary')

validation_generator = datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_width, img_height),
        batch_size=32,
        class_mode='binary')


Found 2048 images belonging to 2 classes.
Found 832 images belonging to 2 classes.

Small Conv Net

Model architecture definition


In [5]:
model = Sequential()

model.add(Conv2D(32,(3,3), input_shape=(img_width, img_height,3)))
#model.add(Convolution2D(32, 3, 3, input_shape=(img_width, img_height,3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32,(3,3)))
#model.add(Convolution2D(32, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64,(3,3)))
#model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

In [6]:
model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

Training


In [7]:
nb_epoch = 30
nb_train_samples = 2048
nb_validation_samples = 832
https://github.com/fchollet/keras/wiki/Keras-2.0-release-notes The methods fit_generator, evaluate_generator and predict_generator now work by drawing a number of batches from a generator (number of training steps), rather than a number of samples. - samples_per_epoch was renamed steps_per_epoch in fit_generator. - nb_val_samples was renamed validation_steps in fit_generator. - val_samples was renamed steps in evaluate_generator and predict_generator.

In [8]:
model.fit_generator(
        train_generator,
        samples_per_epoch=nb_train_samples,
        nb_epoch=nb_epoch,
        validation_data=validation_generator,
        nb_val_samples=nb_validation_samples)


/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:6: UserWarning: Update your `fit_generator` call to the Keras 2 API: `fit_generator(<keras.pre..., validation_data=<keras.pre..., steps_per_epoch=128, epochs=30, validation_steps=832)`
  
Epoch 1/30
128/128 [==============================] - 225s - loss: 0.7594 - acc: 0.5063 - val_loss: 0.6779 - val_acc: 0.6647
Epoch 2/30
128/128 [==============================] - 227s - loss: 0.6774 - acc: 0.5938 - val_loss: 0.6206 - val_acc: 0.6838
Epoch 3/30
128/128 [==============================] - 225s - loss: 0.6110 - acc: 0.6660 - val_loss: 0.5794 - val_acc: 0.7079
Epoch 4/30
128/128 [==============================] - 224s - loss: 0.5657 - acc: 0.7080 - val_loss: 0.5663 - val_acc: 0.6932
Epoch 5/30
128/128 [==============================] - 223s - loss: 0.5278 - acc: 0.7500 - val_loss: 0.5542 - val_acc: 0.7094
Epoch 6/30
128/128 [==============================] - 225s - loss: 0.4792 - acc: 0.7622 - val_loss: 0.5414 - val_acc: 0.7243
Epoch 7/30
128/128 [==============================] - 220s - loss: 0.4314 - acc: 0.7993 - val_loss: 0.5875 - val_acc: 0.7079
Epoch 8/30
128/128 [==============================] - 214s - loss: 0.3890 - acc: 0.8276 - val_loss: 0.6301 - val_acc: 0.7363
Epoch 9/30
128/128 [==============================] - 219s - loss: 0.3368 - acc: 0.8394 - val_loss: 0.6083 - val_acc: 0.7070
Epoch 10/30
128/128 [==============================] - 218s - loss: 0.3029 - acc: 0.8633 - val_loss: 0.6258 - val_acc: 0.7503
Epoch 11/30
128/128 [==============================] - 215s - loss: 0.2719 - acc: 0.8853 - val_loss: 0.6840 - val_acc: 0.7427
Epoch 12/30
128/128 [==============================] - 213s - loss: 0.2329 - acc: 0.9077 - val_loss: 0.7393 - val_acc: 0.7184
Epoch 13/30
128/128 [==============================] - 224s - loss: 0.1950 - acc: 0.9199 - val_loss: 0.9375 - val_acc: 0.7272
Epoch 14/30
128/128 [==============================] - 215s - loss: 0.1785 - acc: 0.9287 - val_loss: 0.8803 - val_acc: 0.7117
Epoch 15/30
128/128 [==============================] - 214s - loss: 0.1586 - acc: 0.9434 - val_loss: 0.9821 - val_acc: 0.7242
Epoch 16/30
128/128 [==============================] - 217s - loss: 0.1487 - acc: 0.9458 - val_loss: 1.2379 - val_acc: 0.7078
Epoch 17/30
128/128 [==============================] - 213s - loss: 0.1350 - acc: 0.9473 - val_loss: 1.3389 - val_acc: 0.7102
Epoch 18/30
128/128 [==============================] - 214s - loss: 0.1288 - acc: 0.9531 - val_loss: 1.2554 - val_acc: 0.7152
Epoch 19/30
128/128 [==============================] - 214s - loss: 0.1107 - acc: 0.9644 - val_loss: 1.4506 - val_acc: 0.7345
Epoch 20/30
128/128 [==============================] - 213s - loss: 0.1305 - acc: 0.9556 - val_loss: 1.8262 - val_acc: 0.7241
Epoch 21/30
128/128 [==============================] - 215s - loss: 0.1099 - acc: 0.9624 - val_loss: 1.2620 - val_acc: 0.6966
Epoch 22/30
128/128 [==============================] - 214s - loss: 0.0973 - acc: 0.9712 - val_loss: 1.5487 - val_acc: 0.7091
Epoch 23/30
128/128 [==============================] - 215s - loss: 0.1015 - acc: 0.9683 - val_loss: 1.8705 - val_acc: 0.7175
Epoch 24/30
128/128 [==============================] - 214s - loss: 0.1582 - acc: 0.9521 - val_loss: 2.6086 - val_acc: 0.6996
Epoch 25/30
128/128 [==============================] - 215s - loss: 0.0975 - acc: 0.9668 - val_loss: 1.9866 - val_acc: 0.7332
Epoch 26/30
128/128 [==============================] - 220s - loss: 0.1310 - acc: 0.9634 - val_loss: 1.6611 - val_acc: 0.7212
Epoch 27/30
128/128 [==============================] - 214s - loss: 0.1281 - acc: 0.9619 - val_loss: 1.9843 - val_acc: 0.7221
Epoch 28/30
128/128 [==============================] - 213s - loss: 0.1326 - acc: 0.9570 - val_loss: 1.8194 - val_acc: 0.7139
Epoch 29/30
128/128 [==============================] - 214s - loss: 0.0875 - acc: 0.9722 - val_loss: 1.4714 - val_acc: 0.7287
Epoch 30/30
128/128 [==============================] - 214s - loss: 0.1298 - acc: 0.9648 - val_loss: 1.8176 - val_acc: 0.7066
Out[8]:
<keras.callbacks.History at 0x7f8d4f8122b0>

In [9]:
model.save_weights('models/basic_cnn_20_epochs.h5')

In [10]:
#model.load_weights('models_trained/basic_cnn_20_epochs.h5')

If your model successfully runs at one epoch, go back and it for 30 epochs by changing nb_epoch above. I was able to get to an val_acc of 0.71 at 30 epochs. A copy of a pretrained network is available in the pretrained folder.

Evaluating on validation set

Computing loss and accuracy :


In [11]:
model.evaluate_generator(validation_generator, nb_validation_samples)


Out[11]:
[1.8119604979964117, 0.70695612980769229]

Evolution of accuracy on training (blue) and validation (green) sets for 1 to 32 epochs :

After ~10 epochs the neural network reach ~70% accuracy. We can witness overfitting, no progress is made over validation set in the next epochs

Data augmentation for improving the model

By applying random transformation to our train set, we artificially enhance our dataset with new unseen images.
This will hopefully reduce overfitting and allows better generalization capability for our network.

Example of data augmentation applied to a picture:


In [11]:
train_datagen_augmented = ImageDataGenerator(
        rescale=1./255,        # normalize pixel values to [0,1]
        shear_range=0.2,       # randomly applies shearing transformation
        zoom_range=0.2,        # randomly applies shearing transformation
        horizontal_flip=True)  # randomly flip the images

# same code as before
train_generator_augmented = train_datagen_augmented.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=32,
        class_mode='binary')


Found 2048 images belonging to 2 classes.

In [12]:
nb_epoch = 30

In [ ]:
model.fit_generator(
        train_generator_augmented,
        samples_per_epoch=nb_train_samples,
        nb_epoch=nb_epoch,
        validation_data=validation_generator,
        nb_val_samples=nb_validation_samples)


/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:6: UserWarning: Update your `fit_generator` call to the Keras 2 API: `fit_generator(<keras.pre..., validation_data=<keras.pre..., steps_per_epoch=64, epochs=30, validation_steps=832)`
  
Epoch 1/30
64/64 [==============================] - 223s - loss: 0.7402 - acc: 0.4893 - val_loss: 0.7037 - val_acc: 0.5000
Epoch 2/30
64/64 [==============================] - 223s - loss: 0.6979 - acc: 0.5562 - val_loss: 0.6404 - val_acc: 0.6537
Epoch 3/30
64/64 [==============================] - 217s - loss: 0.6553 - acc: 0.6284 - val_loss: 0.6134 - val_acc: 0.6622
Epoch 4/30
64/64 [==============================] - 216s - loss: 0.6316 - acc: 0.6567 - val_loss: 0.5952 - val_acc: 0.6670
Epoch 5/30
64/64 [==============================] - 226s - loss: 0.6010 - acc: 0.6836 - val_loss: 0.5674 - val_acc: 0.7023
Epoch 6/30
64/64 [==============================] - 216s - loss: 0.5728 - acc: 0.7163 - val_loss: 0.5563 - val_acc: 0.6995
Epoch 7/30
64/64 [==============================] - 214s - loss: 0.5706 - acc: 0.7065 - val_loss: 0.5613 - val_acc: 0.7082
Epoch 8/30
64/64 [==============================] - 215s - loss: 0.5460 - acc: 0.7241 - val_loss: 0.5535 - val_acc: 0.6980
Epoch 9/30
64/64 [==============================] - 215s - loss: 0.5528 - acc: 0.7334 - val_loss: 0.5998 - val_acc: 0.6798
Epoch 10/30
64/64 [==============================] - 215s - loss: 0.5402 - acc: 0.7412 - val_loss: 0.5203 - val_acc: 0.7370
Epoch 11/30
64/64 [==============================] - 215s - loss: 0.5292 - acc: 0.7280 - val_loss: 0.5103 - val_acc: 0.7465
Epoch 12/30
64/64 [==============================] - 215s - loss: 0.5077 - acc: 0.7612 - val_loss: 0.4883 - val_acc: 0.7560
Epoch 13/30
64/64 [==============================] - 215s - loss: 0.5237 - acc: 0.7510 - val_loss: 0.5166 - val_acc: 0.7402
Epoch 14/30
63/64 [============================>.] - ETA: 0s - loss: 0.4967 - acc: 0.7644
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-13-0adc65d6a700> in <module>()
      4         nb_epoch=nb_epoch,
      5         validation_data=validation_generator,
----> 6         nb_val_samples=nb_validation_samples)

/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     86                 warnings.warn('Update your `' + object_name +
     87                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 88             return func(*args, **kwargs)
     89         wrapper._legacy_support_signature = inspect.getargspec(func)
     90         return wrapper

/opt/conda/lib/python3.6/site-packages/keras/models.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_q_size, workers, pickle_safe, initial_epoch)
   1122                                         workers=workers,
   1123                                         pickle_safe=pickle_safe,
-> 1124                                         initial_epoch=initial_epoch)
   1125 
   1126     @interfaces.legacy_generator_methods_support

/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     86                 warnings.warn('Update your `' + object_name +
     87                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 88             return func(*args, **kwargs)
     89         wrapper._legacy_support_signature = inspect.getargspec(func)
     90         return wrapper

/opt/conda/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_q_size, workers, pickle_safe, initial_epoch)
   1922                                 max_q_size=max_q_size,
   1923                                 workers=workers,
-> 1924                                 pickle_safe=pickle_safe)
   1925                         else:
   1926                             # No need for try/except because

/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     86                 warnings.warn('Update your `' + object_name +
     87                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 88             return func(*args, **kwargs)
     89         wrapper._legacy_support_signature = inspect.getargspec(func)
     90         return wrapper

/opt/conda/lib/python3.6/site-packages/keras/engine/training.py in evaluate_generator(self, generator, steps, max_q_size, workers, pickle_safe)
   2019                                      'or (x, y). Found: ' +
   2020                                      str(generator_output))
-> 2021                 outs = self.test_on_batch(x, y, sample_weight=sample_weight)
   2022 
   2023                 if isinstance(x, list):

/opt/conda/lib/python3.6/site-packages/keras/engine/training.py in test_on_batch(self, x, y, sample_weight)
   1682             ins = x + y + sample_weights
   1683         self._make_test_function()
-> 1684         outputs = self.test_function(ins)
   1685         if len(outputs) == 1:
   1686             return outputs[0]

/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in __call__(self, inputs)
   2267         updated = session.run(self.outputs + [self.updates_op],
   2268                               feed_dict=feed_dict,
-> 2269                               **self.session_kwargs)
   2270         return updated[:len(self.outputs)]
   2271 

/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    787     try:
    788       result = self._run(None, fetches, feed_dict, options_ptr,
--> 789                          run_metadata_ptr)
    790       if run_metadata:
    791         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    995     if final_fetches or final_targets:
    996       results = self._do_run(handle, final_targets, final_fetches,
--> 997                              feed_dict_string, options, run_metadata)
    998     else:
    999       results = []

/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
   1130     if handle is None:
   1131       return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
-> 1132                            target_list, options, run_metadata)
   1133     else:
   1134       return self._do_call(_prun_fn, self._session, handle, feed_dict,

/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1137   def _do_call(self, fn, *args):
   1138     try:
-> 1139       return fn(*args)
   1140     except errors.OpError as e:
   1141       message = compat.as_text(e.message)

/opt/conda/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
   1119         return tf_session.TF_Run(session, options,
   1120                                  feed_dict, fetch_list, target_list,
-> 1121                                  status, run_metadata)
   1122 
   1123     def _prun_fn(session, handle, feed_dict, fetch_list):

KeyboardInterrupt: 

In [ ]:
model.save_weights('models/augmented_30_epochs.h5')

In [15]:
#model.load_weights('models_trained/augmented_30_epochs.h5')

Evaluating on validation set

Computing loss and accuracy :


In [16]:
model.evaluate_generator(validation_generator, nb_validation_samples)


Out[16]:
[0.69096329235113585, 0.76802884615384615]

Evolution of accuracy on training (blue) and validation (green) sets for 1 to 100 epochs :

Thanks to data-augmentation, the accuracy on the validation set improved to ~80%

Using a pre-trained model

The process of training a convolutionnal neural network can be very time-consuming and require a lot of datas.

We can go beyond the previous models in terms of performance and efficiency by using a general-purpose, pre-trained image classifier. This example uses VGG16, a model trained on the ImageNet dataset - which contains millions of images classified in 1000 categories.

On top of it, we add a small multi-layer perceptron and we train it on our dataset.

VGG16 + small MLP

VGG16 model architecture definition


In [17]:
model_vgg = Sequential()
model_vgg.add(ZeroPadding2D((1, 1), input_shape=(img_width, img_height,3)))
model_vgg.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_2'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_2'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_2'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_3'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_2'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_3'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_2'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_3'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

Loading VGG16 weights

This part is a bit complicated because the structure of our model is not exactly the same as the one used when training weights.
Otherwise, we would use the model.load_weights() method.

Note : the VGG16 weights file (~500MB) is not included in this repository. You can download from here :
https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3


In [18]:
import h5py
f = h5py.File('models/vgg/vgg16_weights.h5')
for k in range(f.attrs['nb_layers']):
    if k >= len(model_vgg.layers) - 1:
        # we don't look at the last two layers in the savefile (fully-connected and activation)
        break
    g = f['layer_{}'.format(k)]
    weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]
    layer = model_vgg.layers[k]

    if layer.__class__.__name__ in ['Convolution1D', 'Convolution2D', 'Convolution3D', 'AtrousConvolution2D']:
        weights[0] = np.transpose(weights[0], (2, 3, 1, 0))

    layer.set_weights(weights)

f.close()

Using the VGG16 model to process samples


In [19]:
train_generator_bottleneck = datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=32,
        class_mode=None,
        shuffle=False)

validation_generator_bottleneck = datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_width, img_height),
        batch_size=32,
        class_mode=None,
        shuffle=False)


Found 2048 images belonging to 2 classes.
Found 832 images belonging to 2 classes.

This is a long process, so we save the output of the VGG16 once and for all.


In [20]:
bottleneck_features_train = model_vgg.predict_generator(train_generator_bottleneck, nb_train_samples)
np.save(open('models/bottleneck_features_train.npy', 'wb'), bottleneck_features_train)

In [21]:
bottleneck_features_validation = model_vgg.predict_generator(validation_generator_bottleneck, nb_validation_samples)
np.save(open('models/bottleneck_features_validation.npy', 'wb'), bottleneck_features_validation)

Now we can load it...


In [22]:
train_data = np.load(open('models/bottleneck_features_train.npy', 'rb'))
train_labels = np.array([0] * (nb_train_samples // 2) + [1] * (nb_train_samples // 2))

validation_data = np.load(open('models/bottleneck_features_validation.npy', 'rb'))
validation_labels = np.array([0] * (nb_validation_samples // 2) + [1] * (nb_validation_samples // 2))

And define and train the custom fully connected neural network :


In [23]:
model_top = Sequential()
model_top.add(Flatten(input_shape=train_data.shape[1:]))
model_top.add(Dense(256, activation='relu'))
model_top.add(Dropout(0.5))
model_top.add(Dense(1, activation='sigmoid'))

model_top.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])

In [24]:
nb_epoch=40
model_top.fit(train_data, train_labels,
          nb_epoch=nb_epoch, batch_size=32,
          validation_data=(validation_data, validation_labels))


Train on 2048 samples, validate on 832 samples
Epoch 1/40
2048/2048 [==============================] - 0s - loss: 0.9784 - acc: 0.6855 - val_loss: 0.4140 - val_acc: 0.8077
Epoch 2/40
2048/2048 [==============================] - 0s - loss: 0.4831 - acc: 0.7891 - val_loss: 0.3646 - val_acc: 0.8365
Epoch 3/40
2048/2048 [==============================] - 0s - loss: 0.3974 - acc: 0.8271 - val_loss: 0.3369 - val_acc: 0.8558
Epoch 4/40
2048/2048 [==============================] - 0s - loss: 0.3438 - acc: 0.8525 - val_loss: 0.3136 - val_acc: 0.8594
Epoch 5/40
2048/2048 [==============================] - 0s - loss: 0.3131 - acc: 0.8647 - val_loss: 0.5777 - val_acc: 0.7873
Epoch 6/40
2048/2048 [==============================] - 0s - loss: 0.2883 - acc: 0.8823 - val_loss: 0.2913 - val_acc: 0.8786
Epoch 7/40
2048/2048 [==============================] - 0s - loss: 0.2321 - acc: 0.9048 - val_loss: 0.3724 - val_acc: 0.8462
Epoch 8/40
2048/2048 [==============================] - 0s - loss: 0.2057 - acc: 0.9092 - val_loss: 0.4276 - val_acc: 0.8438
Epoch 9/40
2048/2048 [==============================] - 0s - loss: 0.2000 - acc: 0.9141 - val_loss: 0.3741 - val_acc: 0.8618
Epoch 10/40
2048/2048 [==============================] - 0s - loss: 0.1811 - acc: 0.9248 - val_loss: 0.3447 - val_acc: 0.8726
Epoch 11/40
2048/2048 [==============================] - 0s - loss: 0.1540 - acc: 0.9395 - val_loss: 0.3352 - val_acc: 0.8714
Epoch 12/40
2048/2048 [==============================] - 0s - loss: 0.1511 - acc: 0.9414 - val_loss: 0.3706 - val_acc: 0.8738
Epoch 13/40
2048/2048 [==============================] - 0s - loss: 0.1362 - acc: 0.9399 - val_loss: 0.3913 - val_acc: 0.8798
Epoch 14/40
2048/2048 [==============================] - 0s - loss: 0.1134 - acc: 0.9497 - val_loss: 0.3854 - val_acc: 0.8726
Epoch 15/40
2048/2048 [==============================] - 0s - loss: 0.1117 - acc: 0.9609 - val_loss: 0.5750 - val_acc: 0.8534
Epoch 16/40
2048/2048 [==============================] - 0s - loss: 0.0998 - acc: 0.9565 - val_loss: 0.4339 - val_acc: 0.8870
Epoch 17/40
2048/2048 [==============================] - 0s - loss: 0.0678 - acc: 0.9736 - val_loss: 0.4881 - val_acc: 0.8762
Epoch 18/40
2048/2048 [==============================] - 0s - loss: 0.0852 - acc: 0.9658 - val_loss: 0.4335 - val_acc: 0.8870
Epoch 19/40
2048/2048 [==============================] - 0s - loss: 0.0733 - acc: 0.9692 - val_loss: 0.4505 - val_acc: 0.8750
Epoch 20/40
2048/2048 [==============================] - 0s - loss: 0.0616 - acc: 0.9756 - val_loss: 0.5825 - val_acc: 0.8606
Epoch 21/40
2048/2048 [==============================] - 0s - loss: 0.0585 - acc: 0.9771 - val_loss: 0.6856 - val_acc: 0.8510
Epoch 22/40
2048/2048 [==============================] - 0s - loss: 0.0494 - acc: 0.9800 - val_loss: 0.5624 - val_acc: 0.8750
Epoch 23/40
2048/2048 [==============================] - 0s - loss: 0.0464 - acc: 0.9839 - val_loss: 0.8560 - val_acc: 0.8401
Epoch 24/40
2048/2048 [==============================] - 0s - loss: 0.0626 - acc: 0.9727 - val_loss: 0.5666 - val_acc: 0.8738
Epoch 25/40
2048/2048 [==============================] - 0s - loss: 0.0285 - acc: 0.9912 - val_loss: 0.5543 - val_acc: 0.8786
Epoch 26/40
2048/2048 [==============================] - 0s - loss: 0.0463 - acc: 0.9805 - val_loss: 0.6137 - val_acc: 0.8738
Epoch 27/40
2048/2048 [==============================] - 0s - loss: 0.0260 - acc: 0.9917 - val_loss: 0.6608 - val_acc: 0.8882
Epoch 28/40
2048/2048 [==============================] - 0s - loss: 0.0414 - acc: 0.9824 - val_loss: 0.6423 - val_acc: 0.8702
Epoch 29/40
2048/2048 [==============================] - 0s - loss: 0.0308 - acc: 0.9878 - val_loss: 0.6941 - val_acc: 0.8726
Epoch 30/40
2048/2048 [==============================] - 0s - loss: 0.0273 - acc: 0.9897 - val_loss: 0.6677 - val_acc: 0.8678
Epoch 31/40
2048/2048 [==============================] - 0s - loss: 0.0315 - acc: 0.9927 - val_loss: 0.6830 - val_acc: 0.8738
Epoch 32/40
2048/2048 [==============================] - 0s - loss: 0.0258 - acc: 0.9922 - val_loss: 0.7593 - val_acc: 0.8798
Epoch 33/40
2048/2048 [==============================] - 0s - loss: 0.0266 - acc: 0.9922 - val_loss: 0.8203 - val_acc: 0.8546
Epoch 34/40
2048/2048 [==============================] - 0s - loss: 0.0198 - acc: 0.9941 - val_loss: 0.9712 - val_acc: 0.8450
Epoch 35/40
2048/2048 [==============================] - 0s - loss: 0.0291 - acc: 0.9902 - val_loss: 0.8274 - val_acc: 0.8594
Epoch 36/40
2048/2048 [==============================] - 0s - loss: 0.0168 - acc: 0.9941 - val_loss: 0.8353 - val_acc: 0.8738
Epoch 37/40
2048/2048 [==============================] - 0s - loss: 0.0324 - acc: 0.9912 - val_loss: 0.7481 - val_acc: 0.8702
Epoch 38/40
2048/2048 [==============================] - 0s - loss: 0.0191 - acc: 0.9932 - val_loss: 0.8011 - val_acc: 0.8750
Epoch 39/40
2048/2048 [==============================] - 0s - loss: 0.0254 - acc: 0.9897 - val_loss: 0.7558 - val_acc: 0.8702
Epoch 40/40
2048/2048 [==============================] - 0s - loss: 0.0267 - acc: 0.9897 - val_loss: 0.7828 - val_acc: 0.8798
Out[24]:
<keras.callbacks.History at 0x7fb0b60feb90>

The training process of this small neural network is very fast : ~2s per epoch


In [25]:
model_top.save_weights('models/bottleneck_40_epochs.h5')

Bottleneck model evaluation


In [26]:
#model_top.load_weights('models/with-bottleneck/1000-samples--100-epochs.h5')
#model_top.load_weights('/notebook/Data1/Code/keras-workshop/models/with-bottleneck/1000-samples--100-epochs.h5')

Loss and accuracy :


In [27]:
model_top.evaluate(validation_data, validation_labels)


704/832 [========================>.....] - ETA: 0s
Out[27]:
[0.782802758165277, 0.87980769230769229]

Evolution of accuracy on training (blue) and validation (green) sets for 1 to 32 epochs :

We reached a 90% accuracy on the validation after ~1m of training (~20 epochs) and 8% of the samples originally available on the Kaggle competition !


In [28]:
##Fine-tuning the top layers of a a pre-trained network

Start by instantiating the VGG base and loading its weights.


In [30]:
model_vgg = Sequential()
model_vgg.add(ZeroPadding2D((1, 1), input_shape=(img_width, img_height,3)))
model_vgg.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_2'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_2'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_2'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_3'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_2'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_3'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_1'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_2'))
model_vgg.add(ZeroPadding2D((1, 1)))
model_vgg.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_3'))
model_vgg.add(MaxPooling2D((2, 2), strides=(2, 2)))

In [31]:
import h5py
f = h5py.File('models/vgg/vgg16_weights.h5')
for k in range(f.attrs['nb_layers']):
    if k >= len(model_vgg.layers) - 1:
        # we don't look at the last two layers in the savefile (fully-connected and activation)
        break
    g = f['layer_{}'.format(k)]
    weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]
    layer = model_vgg.layers[k]

    if layer.__class__.__name__ in ['Convolution1D', 'Convolution2D', 'Convolution3D', 'AtrousConvolution2D']:
        weights[0] = np.transpose(weights[0], (2, 3, 1, 0))

    layer.set_weights(weights)

f.close()

Build a classifier model to put on top of the convolutional model. For the fine tuning, we start with a fully trained-classifer. We will use the weights from the earlier model. And then we will add this model on top of the convolutional base.


In [32]:
top_model = Sequential()
top_model.add(Flatten(input_shape=model_vgg.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(1, activation='sigmoid'))

top_model.load_weights('models/bottleneck_40_epochs.h5')

model_vgg.add(top_model)

For fine turning, we only want to train a few layers. This line will set the first 25 layers (up to the conv block) to non-trainable.


In [33]:
for layer in model_vgg.layers[:25]:
    layer.trainable = False

In [34]:
# compile the model with a SGD/momentum optimizer
# and a very slow learning rate.
model_vgg.compile(loss='binary_crossentropy',
              optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

In [35]:
# prepare data augmentation configuration  . . . do we need this?
train_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_height, img_width),
        batch_size=32,
        class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_height, img_width),
        batch_size=32,
        class_mode='binary')


Found 2048 images belonging to 2 classes.
Found 832 images belonging to 2 classes.

In [36]:
# fine-tune the model
model_vgg.fit_generator(
        train_generator,
        samples_per_epoch=nb_train_samples,
        nb_epoch=nb_epoch,
        validation_data=validation_generator,
        nb_val_samples=nb_validation_samples)


Epoch 1/40
2048/2048 [==============================] - 39s - loss: 0.3593 - acc: 0.8862 - val_loss: 0.3669 - val_acc: 0.8738
Epoch 2/40
2048/2048 [==============================] - 35s - loss: 0.2559 - acc: 0.9077 - val_loss: 0.3483 - val_acc: 0.8594
Epoch 3/40
2048/2048 [==============================] - 35s - loss: 0.1942 - acc: 0.9292 - val_loss: 0.3380 - val_acc: 0.8618
Epoch 4/40
2048/2048 [==============================] - 35s - loss: 0.1900 - acc: 0.9258 - val_loss: 0.3268 - val_acc: 0.8858
Epoch 5/40
2048/2048 [==============================] - 35s - loss: 0.1638 - acc: 0.9395 - val_loss: 0.3094 - val_acc: 0.8894
Epoch 6/40
2048/2048 [==============================] - 35s - loss: 0.1307 - acc: 0.9502 - val_loss: 0.3038 - val_acc: 0.8930
Epoch 7/40
2048/2048 [==============================] - 35s - loss: 0.1223 - acc: 0.9556 - val_loss: 0.3267 - val_acc: 0.8990
Epoch 8/40
2048/2048 [==============================] - 35s - loss: 0.1233 - acc: 0.9570 - val_loss: 0.2864 - val_acc: 0.8930
Epoch 9/40
2048/2048 [==============================] - 35s - loss: 0.1147 - acc: 0.9648 - val_loss: 0.3657 - val_acc: 0.8990
Epoch 10/40
2048/2048 [==============================] - 35s - loss: 0.1070 - acc: 0.9590 - val_loss: 0.3192 - val_acc: 0.8990
Epoch 11/40
2048/2048 [==============================] - 35s - loss: 0.0825 - acc: 0.9683 - val_loss: 0.4118 - val_acc: 0.8990
Epoch 12/40
2048/2048 [==============================] - 35s - loss: 0.0953 - acc: 0.9663 - val_loss: 0.3561 - val_acc: 0.9002
Epoch 13/40
2048/2048 [==============================] - 35s - loss: 0.0605 - acc: 0.9756 - val_loss: 0.3643 - val_acc: 0.9026
Epoch 14/40
2048/2048 [==============================] - 35s - loss: 0.0826 - acc: 0.9736 - val_loss: 0.3549 - val_acc: 0.8990
Epoch 15/40
2048/2048 [==============================] - 35s - loss: 0.0758 - acc: 0.9707 - val_loss: 0.3911 - val_acc: 0.9014
Epoch 16/40
2048/2048 [==============================] - 35s - loss: 0.0547 - acc: 0.9756 - val_loss: 0.4181 - val_acc: 0.9002
Epoch 17/40
2048/2048 [==============================] - 35s - loss: 0.0573 - acc: 0.9795 - val_loss: 0.3912 - val_acc: 0.9038
Epoch 18/40
2048/2048 [==============================] - 35s - loss: 0.0499 - acc: 0.9810 - val_loss: 0.4097 - val_acc: 0.9075
Epoch 19/40
2048/2048 [==============================] - 35s - loss: 0.0504 - acc: 0.9790 - val_loss: 0.4063 - val_acc: 0.9159
Epoch 20/40
2048/2048 [==============================] - 35s - loss: 0.0398 - acc: 0.9858 - val_loss: 0.4282 - val_acc: 0.9099
Epoch 21/40
2048/2048 [==============================] - 35s - loss: 0.0477 - acc: 0.9824 - val_loss: 0.4364 - val_acc: 0.8930
Epoch 22/40
2048/2048 [==============================] - 35s - loss: 0.0401 - acc: 0.9873 - val_loss: 0.4338 - val_acc: 0.9038
Epoch 23/40
2048/2048 [==============================] - 35s - loss: 0.0357 - acc: 0.9897 - val_loss: 0.4830 - val_acc: 0.9026
Epoch 24/40
2048/2048 [==============================] - 35s - loss: 0.0397 - acc: 0.9878 - val_loss: 0.4229 - val_acc: 0.9111
Epoch 25/40
2048/2048 [==============================] - 35s - loss: 0.0353 - acc: 0.9863 - val_loss: 0.4715 - val_acc: 0.9171
Epoch 26/40
2048/2048 [==============================] - 35s - loss: 0.0308 - acc: 0.9888 - val_loss: 0.4751 - val_acc: 0.9159
Epoch 27/40
2048/2048 [==============================] - 35s - loss: 0.0432 - acc: 0.9834 - val_loss: 0.4557 - val_acc: 0.9099
Epoch 28/40
2048/2048 [==============================] - 35s - loss: 0.0385 - acc: 0.9868 - val_loss: 0.4859 - val_acc: 0.9038
Epoch 29/40
2048/2048 [==============================] - 35s - loss: 0.0270 - acc: 0.9893 - val_loss: 0.4334 - val_acc: 0.9062
Epoch 30/40
2048/2048 [==============================] - 35s - loss: 0.0356 - acc: 0.9897 - val_loss: 0.4468 - val_acc: 0.9038
Epoch 31/40
2048/2048 [==============================] - 35s - loss: 0.0344 - acc: 0.9888 - val_loss: 0.4405 - val_acc: 0.9087
Epoch 32/40
2048/2048 [==============================] - 35s - loss: 0.0273 - acc: 0.9897 - val_loss: 0.5071 - val_acc: 0.9050
Epoch 33/40
2048/2048 [==============================] - 35s - loss: 0.0179 - acc: 0.9937 - val_loss: 0.5163 - val_acc: 0.9111
Epoch 34/40
2048/2048 [==============================] - 35s - loss: 0.0397 - acc: 0.9868 - val_loss: 0.4168 - val_acc: 0.9123
Epoch 35/40
2048/2048 [==============================] - 35s - loss: 0.0240 - acc: 0.9941 - val_loss: 0.4653 - val_acc: 0.9111
Epoch 36/40
2048/2048 [==============================] - 35s - loss: 0.0206 - acc: 0.9932 - val_loss: 0.5174 - val_acc: 0.9062
Epoch 37/40
2048/2048 [==============================] - 35s - loss: 0.0308 - acc: 0.9902 - val_loss: 0.5016 - val_acc: 0.9087
Epoch 38/40
2048/2048 [==============================] - 35s - loss: 0.0159 - acc: 0.9966 - val_loss: 0.4876 - val_acc: 0.9111
Epoch 39/40
2048/2048 [==============================] - 35s - loss: 0.0335 - acc: 0.9883 - val_loss: 0.4280 - val_acc: 0.9050
Epoch 40/40
2048/2048 [==============================] - 35s - loss: 0.0198 - acc: 0.9922 - val_loss: 0.4693 - val_acc: 0.9038
Out[36]:
<keras.callbacks.History at 0x7fb059495c50>

In [37]:
model_vgg.save_weights('models/finetuning_20epochs_vgg.h5')

In [38]:
model_vgg.load_weights('models/finetuning_20epochs_vgg.h5')

Evaluating on validation set

Computing loss and accuracy :


In [39]:
model_vgg.evaluate_generator(validation_generator, nb_validation_samples)


Out[39]:
[0.46926221093879295, 0.90384615384615385]

In [40]:
model.evaluate_generator(validation_generator, nb_validation_samples)


Out[40]:
[0.69096328031558252, 0.76802884615384615]

In [41]:
model_top.evaluate(validation_data, validation_labels)


704/832 [========================>.....] - ETA: 0s
Out[41]:
[0.782802758165277, 0.87980769230769229]